Systems and methods for displaying predetermined information for clinical use

ABSTRACT

Systems and methods for display of information for clinical use. The method is executable by a processor of a computer system and comprises: (i) receiving data, by the processor, from a communication network comprising: a first reader, a second reader and a wireless sensor network (WSN) router, (ii) based on the received data, determining at least one of a position and an orientation of a given element of the plurality of elements, (iii) in response to the determined at least one of the position and the orientation, causing to be displayed, on a display communicatively coupled to the computer system, predetermined information associated with the determined at least one of a position and an orientation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is the first application filed for the instantly disclosedtechnology.

TECHNICAL FIELD

The present disclosure generally relates to computer implemented methodsand systems for causing a display of information for clinical use, andparticularly, although not exclusively, wherein the display is anaugmented reality or a mixed reality based virtual display overlaid on areal environment.

BACKGROUND

Typically, in clinical environments such as operating rooms (ORs), oneor more users (e.g., clinician, surgeon, surgical assistant, nurse,patient, trainee clinician, etc.) must refer to many different types ofvisual patient information during a medical procedure. Visual patientinformation may include images and videos of the patient or body partsof the patient acquired by imaging equipment, such as one or more ofmagnetic resonance imaging equipment, computed tomography scanningequipment, x-ray equipment, three-dimensional (3D) ultrasound equipment,endoscopic equipment, 3D computer modeling equipment, patient monitoringequipment. Visual patient information may also include test results,such as those printed on paper, or displayed on a screen as retrievedfrom a medical records database. As well, visual information may alsoinclude procedure-related information, such as those printed on paper,or displayed on a screen as retrieved from surgical/medical techniqueprotocol database The visual patient information assists the one or moreuser (such as the surgeon and the surgical assistant) to plan for andexecute the medical procedure.

ORs therefore typically include many displays positioned at variouslocations for displaying the visual patient information during themedical procedure to the users. For example, there may be one or moredisplays hung from a ceiling, mounted on a wall, and/or supported on acart of the OR. However, positioning all the displays for convenientviewing by all necessary users at all time points during the medicalprocedure is difficult, if not impossible. Furthermore, the necessity,for a single user, to frequently shift their focal reference betweenremotely located multiple displays as well as the patient often cancause fatigue in that user and have a deleterious effect on quality ofthe procedure. Moreover, a given display of the patient information mayrequire mental translation between an orientation of an anatomy of thepatient information shown on the display and the orientation of theactual anatomy of the patient can be particularly difficult, prone toerrors, or inefficiently time-consuming for the users during the medicalprocedure.

Finally, locating the precise location on the patient from the patientinformation on the display can also be difficult. For example, it can bedifficult for the user to identify where a particular location within adisplayed x-ray or other image corresponds to on the patient. Moreover,the user may have used a 3D anatomical model to practice before themedical procedure, but may not be able to effectively use the modelduring the medical procedure because of the inherent difficulty ofrelating the model to the patient in real-time.

It is desired to provide alternative and/or improved methods and systemsfor causing display of information for clinical use.

SUMMARY

The embodiments of the present disclosure have been developed based onDevelopers' appreciation of shortcomings associated with the prior art.Developers have noted that augmented reality and mixed reality methodsand systems exist to overlay patient information over the realenvironment during a medical procedure or in a clinical setting. Suchvirtual display methods and systems can obviate some of the problemsnoted above such as the need to view different displays in differentlocations. The augmented reality and/or mixed reality methods andsystems of the prior art utilize markers attached to an object, such asthe patient, a medical instrument or the user, to track a movement ofthe object. The tracked movement is then used as a trigger for thedisplay of information on a display associated with the user.

However, Developers have noted that such prior art systems using virtualdisplay overlays suffer from lack of sufficient accuracy, stability andreliability as they use conventional communication techniques totransmit data from various markers to a computer system which receivesdata from the markers, tracks their movement and causes the virtualdisplay.

In particular, Developers have noted that the conventional communicationtechniques rely on a single communication route for data transmissionfrom the markers to the computer system. In an event of disruption ofthe communication route, the conventional communication techniques mayencounter several operational failures. During a medical/surgicalprocedure, this can have catastrophic if not fatal outcomes.

To overcome such shortcomings, Developers of the present technology havedevised systems and methods for displaying information for clinical useusing augmented reality or mixed reality technologies, for example, andwhich rely on more than one communication route between markers and acomputer system.

The systems and methods of the present technology comprise multipleelements (markers) which can transmit data to a computer system throughat least two communication routes, and in some embodiments, throughthree or four communication routes. Data transmission through multipleroutes therefore can provide back-up in case one communication route isdisrupted, thereby ensuring a reliable communication link.

In embodiments of the present technology, different communication routesare broadly provided by utilizing more than one reader, each configuredto receive data from each element. In other words, embodiments of thepresent technology rely on at least two readers, such as a first readerand a second reader, that independently communicate with the pluralityof elements to receive the data and to directly communicate the data viaa communication link to the computer system. In addition, in certainnon-limiting embodiments, the first reader and the second reader may beconfigured to wirelessly communicate with the computer system via awireless sensor network (WSN) router.

Thus, providing more than one reader, such as the first reader and thesecond reader, instead of a single reader, and optionally an additionalcommunication layer of the WSN router may provide multiple copies of thedata from the elements to the computer system for processing and therebyensure a more accurate, stable and reliable system.

In accordance with the first broad aspect of the present disclosure,there is provided a method for displaying information for clinical use,the method executable by a processor of a computer system, the methodcomprising: receiving data, by the processor, from a communicationnetwork comprising: a first reader, a second reader and a wirelesssensor network (WSN) router: the first reader being communicativelycoupled to, and configured to receive data from, each element of aplurality of elements, each element positioned at a given locationrelative to an object at a clinical site; the second reader beingcommunicatively coupled to, and configured to receive data from, eachelement of the plurality of elements; the WSN router communicativelycoupled to, and configured to receive data from, each of the firstreader and the second reader; based on the received data, determining atleast one of a position and an orientation of a given element of theplurality of elements; in response to the determined at least one of theposition and the orientation, causing to be displayed, on a displaycommunicatively coupled to the computer system, predeterminedinformation associated with the determined at least one of a positionand an orientation.

In accordance with certain embodiments of the present disclosure, themethod further comprises: determining if the data has been received frommore than one of: the first reader, the second reader and the WSNrouter; in response to the data having been received from more than oneof the first reader, the second reader and the WSN router, filtering thedata to obtain filtered data, the filtering comprising disregardingduplicate data based on a predetermined rule, the determining the atleast one of the position and the orientation being based on thefiltered data.

In accordance with certain embodiments of the present disclosure, thepredetermined rule comprises one of: a predetermined hierarchy ofcommunication routes between the plurality of elements and the computersystem, a predetermined hierarchy of time of transmission of the data;and a predetermined hierarchy of time of reception of the data by thecomputer system.

In accordance with certain embodiments of the present disclosure, theobject is one or more of: a patient, a medical/surgical instrument, andoperating/procedural table.

In accordance with certain embodiments of the present disclosure,determining at least one of the position and the orientation of thegiven element of the plurality of elements comprises triangulation.

In accordance with certain embodiments the present disclosure,triangulation is performed by computing angles of incidences between theplurality of elements and the first reader and the second reader.

In accordance with certain embodiments of the present disclosure, atleast one of the position and the orientation of the given element ofthe plurality of elements is determined based on a 3D point cloudrepresentation of region of interest (ROI).

In accordance with certain embodiments of the present disclosure, thedisplay is a fixed display or is part of a wearable device. The wearabledevice may include but not be limited to a headset, glasses or contactlenses. The display may be configured to project a holographicprojection.

In accordance with certain embodiments of the present disclosure, thepredetermined information is one or more of: an augmented image, amedical/surgical procedure guidance, a navigation menu, an eye fatigueindicator, a user's profile, and an annotation.

In accordance with certain embodiments of the present disclosure, thedisplayed information comprises an actionable digital element, based onan input received from a user, the actionable digital element isconfigured to display of further information associated with thepredetermined information.

In accordance with certain embodiments of the present disclosure, theprocessor is configured to retrieve the predetermined information from amemory of the computer system.

In accordance with certain embodiments of the present disclosure, thepredetermined information which is displayed is based on a user'sprofile.

In accordance with certain embodiments of the present disclosure, thedisplayed information comprises a virtual display created by anaugmented reality or a mixed reality method.

In accordance with certain embodiments of the present disclosure,determining if the data was received from more than one of the firstreader, the second reader and the WSN router comprises the processorinterrogating the first reader, the second reader and the wirelesssensor network (WSN) router to determine if any one of communicationroutes between the plurality of elements and the computer system wascompromised.

In accordance with certain embodiments of the present disclosure, thecommunication network defines a plurality of communication routesbetween each of the plurality of elements and the computer system. Incertain embodiments, the communication routes include: (i) between eachof the plurality of elements, the first reader and the computer system,(ii) between each of the plurality of elements, the second reader, andthe computer system, (iii) between each of the plurality of elements,the first reader, the WSN router, and the computer system, and (iv)between each of the plurality of elements, the second reader, the WSNrouter, and the computer system.

In accordance with certain embodiments of the present disclosure, themethod further comprises tracking user's eye, head or arm movement forinteraction with the display.

In accordance with certain embodiments of the present disclosure, the atleast some of the plurality of elements are radio frequency basedelements. In certain embodiments, all of the plurality of elements areradio frequency based elements.

In accordance with certain embodiments of the present disclosure, atleast some of the plurality of elements are optical based elements.

In accordance with a second broad aspect of the present disclosure,there is provided a system for generating a display of information forclinical use, the system comprising a communication network including: aplurality of elements having data stored therein and positioned at agiven location relative to an object at a clinical site; a first readerand a second reader, each of the first reader and the second readercommunicatively coupled to each element of the plurality of elements andconfigured to receive the data from each element of the plurality ofelements and transmit the data to a computer system; a wireless sensornetwork (WSN) router configured to receive the data from the firstreader and the second reader and transmit the data to the computersystem; the computer system being independently communicativelycoupleable with each of the first reader, the second reader and the WSNrouter for receiving the transmitted data independently from each of thefirst reader, the second reader and the WSN router; wherein the computersystem is configured to execute a method comprising: receiving data fromthe communication network; based on the received data, determining atleast one of a position and an orientation of a given element of theplurality of elements; in response to the at least one of the determinedposition and the orientation, causing display, on a displaycommunicatively coupled to the computer system, predeterminedinformation associated with the determined the at least one of adetermined position and an orientation.

In accordance with certain embodiments of the present disclosure, the atleast some of the plurality of elements are radio frequency basedelements.

In accordance with certain embodiments of the present disclosure, atleast some of the plurality of elements are optical based elements.

In accordance with certain embodiments of the present disclosure, thedisplay is incorporated in a wearable device that includes, but is notlimited to headgear, glasses or contact lenses, or the display is afixed display associated with an electronic device such as mobiledevice. The display may be configured to project a holographicprojection.

In accordance with a third broad aspect of the present disclosure, thereis provided a method for displaying information for clinical use, themethod executable by a processor of a computer system, the methodcomprising: receiving data, by the processor, from a communicationnetwork comprising: a first reader and a second reader: the first readerbeing communicatively coupled to, and configured to receive data from,each element of a plurality of elements, each element positioned at agiven location relative to an object at a clinical site; the secondreader being communicatively coupled to, and configured to receive datafrom, each element of the plurality of elements, each element positionedat a given location relative to an object at a clinical site;determining, by the processor, if the data has been received from one ormore of the first reader and the second reader; in response to the datahaving been received from more than one of the first reader and thesecond reader, filtering the data to obtain filtered data, the filteringcomprising disregarding duplicate data; determining, based on thefiltered data, at least one of a position and an orientation of a givenelement of the plurality of elements; in response to the determined atleast one of the position and the orientation, cause to be displayed, ona display communicatively coupled to the computer system, predeterminedinformation associated with the determined at least one of the positionand the orientation.

In accordance with certain embodiments of the present disclosure, thecommunication network further comprises a wireless sensor network (WSN)router, the WSN router communicatively coupled to, and configured toreceive data from, each of the first reader and the second reader, themethod further comprising receiving data from the WSN router.

In accordance with certain embodiments of the present disclosure, themethod further comprises: determining if the data has been received frommore than one of: the first reader, and the second reader; in responseto the data having been received from more than one of the first readerand the second reader, filtering the data to obtain filtered data, thefiltering comprising disregarding duplicate data based on apredetermined rule, the determining the at least one of the position andthe orientation being based on the filtered data.

In accordance with certain embodiments of the present disclosure, thepredetermined rule comprises one of: a predetermined hierarchy ofcommunication routes between the plurality of elements and the computersystem, a predetermined hierarchy of time of transmission of the data;and a predetermined hierarchy of time of reception of the data by thecomputer system.

In accordance with certain embodiments of the present disclosure, theobject is one or more of: a patient, a medical/surgical instrument, andan operating/procedural table.

In accordance with certain embodiments of the present disclosure,determining at least one of the position and the orientation of thegiven element of the plurality of elements comprises triangulation.

In accordance with certain embodiments the present disclosure,triangulation is performed by computing angles of incidences between theplurality of elements and the first reader and the second reader.

In accordance with certain embodiments of the present disclosure, atleast one of the position and the orientation of the given element ofthe plurality of elements is determined based on a 3D point cloudrepresentation of region of interest (ROI).

In accordance with certain embodiments of the present disclosure, thedisplay is a fixed display or is part of a wearable device. The wearabledevice may include, but is not limited to be a headset, glasses orcontact lenses. The display may be configured to project a holographicprojection.

In accordance with certain embodiments of the present disclosure, thepredetermined information is one or more of: an augmented/virtual image,a medical/surgical procedure guidance, a navigation menu, an eye fatigueindicator, a user's profile, and an annotation.

In accordance with certain embodiments of the present disclosure, thedisplayed information comprises an actionable digital element, based onan input received from a user, the actionable digital element isconfigured to display further information associated with thepredetermined information.

In accordance with certain embodiments of the present disclosure, theprocessor is configured to retrieve the predetermined information from amemory of the computer system.

In accordance with certain embodiments of the present disclosure, thepredetermined information which is displayed is based on a user'sprofile.

In accordance with certain embodiments of the present disclosure, thedisplayed information comprises a virtual display created by anaugmented reality or a mixed reality method.

In accordance with certain embodiments of the present disclosure,determining if the data was received from more than one of the firstreader and the second reader comprises the processor interrogating thefirst reader and the second reader to determine if any one ofcommunication routes between each of the plurality of elements and thecomputer system was compromised.

In accordance with certain embodiments of the present disclosure, thecommunication network defines a plurality of communication routesbetween the plurality of elements and the computer system. In certainembodiments, the communication routes include: (i) between each of theplurality of elements, the first reader and the computer system, and(ii) between each of the plurality of elements, the second reader, andthe computer system.

In accordance with certain embodiments of the present disclosure, themethod further comprises tracking user's eye, head or arm movement forinteraction with the display.

In accordance with certain embodiments of the present disclosure, the atleast some of the plurality of elements are radio frequency basedelements. In certain embodiments, all of the plurality of elements areradio frequency based elements.

In accordance with certain embodiments of the present disclosure, atleast some of the plurality of elements are optical based elements.

In accordance with a fourth broad aspect of the present disclosure,there is provided a system for generating a display of information forclinical use, the system comprising a communication network including: aplurality of elements having data stored therein and positioned at agiven location relative to an object at a clinical site; a first readerand a second reader, each of the first reader and the second readercommunicatively coupled to each element of the plurality of elements andconfigured to receive the data from each element of the plurality ofelements and transmit the data to a computer system; the computer systembeing independently communicatively coupleable with each of the firstreader, and the second reader for receiving the transmitted dataindependently from each of the first reader and the second reader;wherein the computer system is configured to execute a methodcomprising: receiving data from the communication network; based on thereceived data, determining at least one of a position and an orientationof a given element of the plurality of elements; in response to the atleast one of the determined position and the orientation, causingdisplay, on a display communicatively coupled to the computer system,predetermined information associated with the determined the at leastone of a determined position and an orientation.

In accordance with certain embodiments of the present disclosure, the atleast some of the plurality of elements are radio frequency basedelements.

In accordance with certain embodiments of the present disclosure, atleast some of the plurality of elements are optical based elements.

In accordance with certain embodiments of the present disclosure, thedisplay is incorporated in a wearable device such as, but not limited toa headgear, glasses or contact lenses, or the display is a fixed displayassociated with an electronic device such as mobile device. The fixeddisplay may include but not limited to holographic projection.

Unless otherwise defined or indicated by context, all technical andscientific terms used herein have the same meaning as commonlyunderstood by one of ordinary skill in the art to which the describedembodiments appertain to.

In the context of the present specification, “computer system” is anycomputer hardware that is capable of running software appropriate to therelevant task at hand. In the context of the present specification, ingeneral the term “computer system” is associated with a user of thecomputer system. Thus, some (non-limiting) examples of computer systemsinclude personal computers (desktops, laptops, netbooks, etc.),smartphones, and tablets, as well as network equipment such as routers,switches, and gateways. It should be noted that a device acting as acomputer system in the present context is not precluded from acting as aserver to other computer systems. The use of the expression “a computersystem” does not preclude multiple computer systems being used inreceiving/sending, carrying out or causing to be carried out any task orrequest, or the consequences of any task or request, or steps of anymethod described herein.

In the context of the present disclosure, the expression “data” includesdata of any nature or kind whatsoever capable of being stored in adatabase. Thus, data includes, but is not limited to, audiovisual works(images, movies, sound records, presentations etc.), data (locationdata, numerical data, etc.), text (opinions, comments, questions,messages, etc.), documents, spreadsheets, etc.

In the context of the present disclosure, unless expressly indicatedotherwise, a “database” is any structured collection of data,irrespective of its particular structure, the database managementsoftware, or the computer hardware on which the data is stored,implemented or otherwise rendered available for use. A database mayreside on the same hardware as the process that stores or makes use ofthe information stored in the database or it may reside on separatehardware, such as a dedicated server or plurality of servers.

In the context of the present disclosure, “communication/communicate”between two nodes, a node and a processor/controller, a node and aserver, a pipeline and a server, any two modules, any two softwarecomponents, or any two hardware components, refers to as the exchange,transfer, sending, receiving, sharing or the like of information,request, data, or the like without limiting the scope of presentdisclosure.

Software modules, modules, or units which are implied to be software,may be represented herein as any combination of flowchart elements orother elements indicating performance of process steps and/or textualdescription. Such modules may be executed by hardware that is expresslyor implicitly shown.

In the context of the present specification, unless provided expresslyotherwise, the words “first”, “second”, “third”, etc. have been used asadjectives only for the purpose of allowing for distinction between thenouns that they modify from one another, and not for the purpose ofdescribing any particular relationship between those nouns. Thus, forexample, it should be understood that, the use of the terms “firstprocessor” and “third processor” is not intended to imply any particularorder, type, chronology, hierarchy or ranking (for example) of/betweenthe server, nor is their use (by itself) intended to imply that any“second server” must necessarily exist in any given situation. Further,as is discussed herein in other contexts, reference to a “first” elementand a “second” element does not preclude the two elements from being thesame actual real-world element. Thus, for example, in some instances, a“first” server and a “second” server may be the same software and/orhardware, in other cases they may be different software and/or hardware.

It will be understood that when an element is referred to as being“connected” or “coupled” to another element, it can be directly orindirectly connected or coupled to the other element or interveningelements that may be present. In contrast, when an element is referredto as being “directly connected” or “directly coupled” to anotherelement, there are no intervening elements present. Other words used todescribe the relationship between elements should be interpreted in alike fashion (e.g., “between” versus “directly between,” “adjacent”versus “directly adjacent,” etc.).

In the context of the present specification, when an element is referredto as being “associated with” another element, in certain embodiments,the two elements can be directly or indirectly linked, related,connected, coupled, the second element employs the first element, or thelike without limiting the scope of present disclosure.

The terminology used herein is only intended to describe particularrepresentative embodiments and is not intended to be limiting of thepresent technology. As used herein, the singular forms “a” “an” and“the” are intended to include the plural forms as well, unless thecontext clearly indicates otherwise. It will be further understood thatthe terms “comprises” and/or “comprising”, when used in thisspecification, specify the presence of stated features, integers, steps,operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof.

Implementations of the present technology each have at least one of theabove-mentioned objects and/or aspects, but do not necessarily have allof them. It should be understood that some aspects of the presenttechnology that have resulted from attempting to attain theabove-mentioned object may not satisfy this object and/or may satisfyother objects not specifically recited herein,

BRIEF DESCRIPTION OF THE FIGURES

Further features and advantages of the present disclosure will becomeapparent from the following detailed description, taken in combinationwith the appended drawings, in which:

FIG. 1 illustrates a system for generating display of information forclinical use, in accordance with various non-limiting embodiments of thepresent disclosure;

FIG. 2A depicts a high-level block diagram of components of a computersystem, in accordance with various embodiments of the presentdisclosure;

FIG. 2B depicts a high-level functional block diagram of a server, inaccordance with various embodiments of the present disclosure;

FIG. 3 illustrates a flowchart of a method implemented on the computersystem of FIG. 2A for generating the display of information for clinicaluse, in accordance with various non-limiting embodiments of the presentdisclosure;

FIG. 4 illustrates at least some of the modules implemented on thecomputer system, in accordance with various non-limited embodiments ofthe present disclosure;

FIG. 5 illustrates a representative technique of triangulation performedby the computer system in accordance with various non-limitingembodiments of the present disclosure;

FIG. 6 illustrates a representative display depicting predeterminedinformation overlaid over a patient's body part in accordance withvarious non-limiting embodiments of the present disclosure; and

FIG. 7 illustrates a flowchart of a method implemented on the computersystem of FIG. 2A for generating the display of information for clinicaluse, in accordance with various alternative non-limiting embodiments ofthe present disclosure.

It is to be understood that throughout the appended drawings andcorresponding descriptions, like features are identified by likereference characters. Furthermore, it is also to be understood that thedrawings and ensuing descriptions are intended for illustrative purposesonly and that such disclosures do not provide a limitation on the scopeof the claims.

DETAILED DESCRIPTION

The instant disclosure is directed to addressing at least some of thedeficiencies of the prior art. In particular, described herein is asystem and method for generating a display of information for clinicaluse, using for example augmented reality and/or mixed realitytechnologies and which is reliant on more than one communication routebetween elements associated with objects in a clinical setting and acomputer system from receiving data from the elements.

The examples and conditional language recited herein are principallyintended to aid the reader in understanding the principles of thepresent technology and not to limit its scope to such specificallyrecited examples and conditions. It will be appreciated that thoseskilled in the art may devise various arrangements which, although notexplicitly described or shown herein, nonetheless embody the principlesof the present technology and are included within its spirit and scope.

Furthermore, as an aid to understanding, the following description maydescribe relatively simplified implementations of the presenttechnology. As persons skilled in the art would understand, variousimplementations of the present technology may be of a greatercomplexity.

In some cases, what are believed to be helpful examples of modificationsto the present technology may also be set forth. This is done merely asan aid to understanding, and, again, not to define the scope or setforth the bounds of the present technology. These modifications are notan exhaustive list, and a person skilled in the art may make othermodifications while nonetheless remaining within the scope of thepresent technology. Further, where no examples of modifications havebeen set forth, it should not be interpreted that no modifications arepossible and/or that what is described is the sole manner ofimplementing that element of the present technology.

Moreover, all statements herein reciting principles, aspects, andimplementations of the present technology, as well as specific examplesthereof, are intended to encompass both structural and functionalequivalents thereof, whether they are currently known or developed inthe future. Thus, for example, it will be appreciated by those skilledin the art that any block diagrams herein represent conceptual views ofillustrative circuitry embodying the principles of the presenttechnology. Similarly, it will be appreciated that any flowcharts, flowdiagrams, state transition diagrams, pseudo-code, and the like representvarious processes which may be substantially represented incomputer-readable media and so executed by a computer or processor,whether or not such computer or processor is explicitly shown.

The functions of the various elements shown in the figures, includingany functional block labeled as a “processor” or a “processing unit”,may be provided through the use of dedicated hardware as well ashardware capable of executing software in association with appropriatesoftware. When provided by a processor, the functions may be provided bya single dedicated processor, by a single shared processor, or by aplurality of individual processors, some of which may be shared. In someembodiments of the present technology, the processor may be ageneral-purpose processor, such as a central processing unit (CPU) or aprocessor dedicated to a specific purpose, such as a graphics processingunit (GPU). Moreover, explicit use of the term “processor” or“controller” should not be construed to refer exclusively to hardwarecapable of executing software, and may implicitly include, withoutlimitation, digital signal processor (DSP) hardware, network processor,application specific integrated circuit (ASIC), field programmable gatearray (FPGA), read-only memory (ROM) for storing software, random accessmemory (RAM), and non-volatile storage. Other hardware, conventionaland/or custom, may also be included.

With these fundamentals in place, and as stated earlier, the instantdisclosure is directed to systems and methods for generating the displayof information for clinical use and which may use augmented reality (AR)and/or mixed reality (MR) technologies to create the displays ofinformation.

MR is a technology used for merging of real and virtual worlds toproduce new environments and visualizations, where physical and digitalobjects co-exist and interact in real time. Mixed reality does notexclusively take place in either the physical world or virtual world,but is a hybrid of reality and virtual reality.

In contrast, the AR technology takes place in the physical world, withinformation or objects added virtually. AR is technology is used fordisplaying virtual or “augmented” objects or visual effects overlaid ona real environment. The real environment may include a room or specificarea or may be more general to include the world at large. The virtualaspects overlaid on the real environment may be represented as anchoredor in a set position relative to one or more aspects of the realenvironment.

It is to be noted that the present technology is not particularlylimited in the technology used to create the display of information,whether AR or MR. The present technology is directed to providing areliable communication link between elements associated with the objectand the computer system, regardless of whether AR, MR or other virtualtechnology is used to display the information.

Embodiments of the systems and methods herein will be described withreference to MR display of virtual components or representations of realobjects overlaid on a real environment and facilitating an interactionof a user with the virtual components. An MR based display allows theuser (e.g., clinician, surgeon, surgical assistant, nurse, patient,trainee clinician, etc.) to view and interact with the displayed virtualobjects that appear to be projected into the real environment, the realenvironment also being visible to the user. In one non-limitingembodiment, the MR based display typically includes two or more displaylenses or screens, including one for each eye of the user. The twodisplay lenses or screens are configured to transmit light such that thereal environment is visible to the user while also projecting thevirtual components or representations of the real objects overlaid tomake visible to the user of the MR based display.

The two or more MR based displays may be used in a coordinated manner,for example with a first MR based display controlling one or moreadditional MR based displays, or in a system with defined roles. Forexample, when activating an MR based display, the user may select a role(e.g., surgeon, surgical assistant, nurse, patient, trainee clinician,etc.) during a surgical procedure or other event and the MR baseddisplay may illustrate information (also referred to herein as“predetermined information”) relevant to that role. In other examples,the user may select a specific event for which the MR display isrequired, such as surgery, therapy, training, patient consultation. Thespecific events or the specific roles are not particularly limited andembodiments of the present technology can be applied to a broad range ofroles and events. The information may comprise predetermined informationrelevant either to the selected event or to a predetermined time withinthat event. In this respect, the predetermined information may comprisesa sequence of predetermined information modules.

FIG. 1 illustrates a system 100 for generating the MR display ofinformation for clinical use, in accordance with various non-limitingembodiments of the present disclosure. As shown, the system 100 mayinclude a plurality of elements 106-1, 106-2, 106-3, and 106-4, readers110-1 and 110-2, a wireless sensor network (WSN) router 112, a computersystem 114, and a display 118. Optionally, the system 100 may include aserver 116. The system 100 may include other components and modules,however such components have been omitted from FIG. 1 for the purpose ofsimplicity.

In certain non-limiting embodiments, the plurality of elements 106-1,106-2, 106-3, and 106-4 may be configured to track one or more objectssuch as tracking a movement of a medical instrument 108, including butnot limited to a scalpel, a clamp, a forcep, scissors, an osteotome, aretractor, leads, dissector, an implant, a filler, or tracking amovement of a patient 104 or a body part of the patient, such as an arm,a leg, a torso, and the like. In certain non-limiting embodiments,different medical/surgical instruments 108 to be tracked and/or trackedtargets such as the patient 104 may be provided with a respective set ofthe plurality of elements 106-1, 106-2, 106-3, and 106-4 in differentconfigurations.

In certain non-limiting embodiments, the plurality of elements 106-1,106-2, 106-3, and 106-4 may be active or passive elements. For example,active elements may include infrared emitters for use with an opticalsensor. Passive elements may include reflective spheres for use with theoptical sensor, or pick-up coils for use with an electromagnetic reader,for example.

In certain non-limiting embodiments, the plurality of elements 106-1,106-2, 106-3, and 106-4 may all be the same type or may include acombination of two or more different types of markers. In somenon-limiting embodiments, the plurality of elements 106-1, 106-2, 106-3,and 106-4 may be radiofrequency (RF) based markers. In othernon-limiting embodiments, the plurality of elements 106-1, 106-2, 106-3,and 106-4 may include but are not limited to reflective markers,electromagnetic (EM) markers, pulsed or un-pulsed light-emitting diode(LED) markers, glass markers, reflective adhesives, or reflective uniquestructures or patterns, among others.

It is to be noted that RF and EM markers may have specific signaturesfor the specific medical/surgical instrument 108 or the patient bodypart they may be attached to or otherwise associated with. Reflectiveadhesives, structures and patterns, glass markers, and LED markers maybe detectable using optical sensors, while RF and EM markers may bedetectable using RF readers and EM readers. Different marker types maybe selected to suit different clinical conditions or other clinicalneeds. For example, using EM and RF markers may enable tracking of themedical instrument 108 without requiring a line-of-sight from thereaders 110-1 and 110-2 to the plurality of elements 106-1, 106-2,106-3, and 106-4, and using optical markers may avoid additional noisefrom electrical emission and detection systems

In some examples, the plurality of elements 106-1, 106-2, 106-3, and106-4 may include printed markers or 3D designs that may be used fordetection by an auxiliary camera, such as a wide-field camera and/or theoptical scope. Printed markers may also be used as a calibrationpattern, for example to provide distance information (e.g., 3D distanceinformation) to the optical sensor. Printed identification markers mayinclude designs such as concentric circles with different ring spacingand/or different types of bar codes, among other designs.

In certain non-limiting embodiments, the plurality of elements 106-1,106-2, 106-3, and 106-4 may have an associated data. In some of theplurality of elements 106-1, 106-2, 106-3, and 106-4, the associateddata may be stored in a memory (in case of active elements). Also, insome of the plurality of elements 106-1, 106-2, 106-3, 106-4, theassociated data may be embedded in the form of visual identification,such as bar codes (in case of passive elements).

In certain non-limiting, the readers 110-1 and 110-2 may becommunicatively coupled to the plurality of elements 106-1, 106-2,106-3, and 106-4 and configured to receive the associated data from eachelement of the plurality of elements 106-1, 106-2, 106-3, and 106-4. Incertain non-limiting embodiments, the associated data may include butnot be limited to previously assigned identifications (IDs), geographiclocations, type of object (e.g., the medical/surgical instrument 108,the patient 104, the operating/procedural table 109 or the like) withwhich the plurality of elements 106-1, 106-2, 106-3, and 106-4 areassociated with or any such data that may assist the computer system 114to cause the information to be displayed on the display 118.

In certain non-limiting embodiments, the readers 110-1 and 110-2 mayinclude RF readers. However, depending on type of the plurality ofelements 106-1, 106-2, 106-3, and 106-4, the readers 110-1 and 110-2 mayalso include EM readers, optical scanners, optical sensors (e.g., aLIDAR or other depth sensor, infrared sensor, color camera, stereoscopiccamera, thermographic camera, multispectral camera).

Further, in certain non-limiting embodiments, the plurality of elements106-1, 106-2, 106-3, and 106-4 may be configured to communicate with theWSN router 112 and the computer system 114 via a communication link 120.The communication link 120 may be based on IEEE 802.11 family ofstandards or any other suitable standard.

Turning now to the display 118, in certain non-limiting embodiments, thedisplay 118 may be a touch-sensitive screen for receiving touch inputs,part of a wearable device (e.g., MR glasses, MR lenses, or a MRheadset), to provide the predetermined information as the MR baseddisplay, a fixed display for displaying still and/or video images (e.g.,a live video image of the surgical field and/or 2D or 3D images obtainedpreoperatively) or any other suitable type of display. In certainnon-limiting embodiments, the display may be configured to projectinteractive holographic projections.

The display 118 may include one or more lenses or screens, such as asingle screen or two screens (e.g., one per eye of the user). Thescreens may allow light to be transmitted therethrough such that aspectsof the real environment are visible while also displaying thepredetermined information such that the predetermined informationappears to be overlaid on the real environment.

The MR based display may be viewable to one or more users and mayinclude differences among views available for the one or more viewerswhile retaining some aspects as universal among the views. For example,a heads-up display may change between two views while the predeterminedinformation may be fixed to a real object or area in both views.Aspects, such as a color of an object, lighting, or other changes may bemade among the views without changing a fixed position of at least onevirtual object.

The display 118 may be used during a medical procedure, for exampleperformed by a surgeon 102 on the patient 104. However, it is to benoted that in certain non-limiting embodiments, the surgeon 102 may bepresent at a remote location from the patient and may assist the medicalprocedure using robotic medical instruments (not illustrated). Thedisplay 118 may present the predetermined information, such as virtualobjects, augmented still/video images, medical procedure guidance,navigation menu, eye fatigue indicator, surgeon's profile, annotationsor the like, details of which will be discussed later in the disclosure,during the medical procedure to augment the surgeon's vision. Thesurgeon 102 may control the predetermined information using a remotecontroller for the display 118, by interacting with the display (e.g.when it is touch screen) or by interacting with the predeterminedinformation (e.g., using a hand or a gesture to “interact” with thepredetermined information). In another example, the predeterminedinformation may be used as a guide for a path of the medical instrument108 relative to the patient 104. In certain examples, the predeterminedinformation may react to movements of other virtual or real-worldobjects in the surgical field. For example, the predeterminedinformation may be altered when the surgeon 102 is manipulating themedical instrument 108 in proximity to the predetermined information.

The system 100 may be implemented in an environment where a medicalprocedure is to be implemented on a patient 104 by a surgeon 102 orother clinician. The medical/surgical procedures may include but are notlimited to orthopaedic procedures, such as but not limited to, bonepreparation, implant positioning, fracture management, osteotomies, limblengthening, deformity correction, spine surgery, tumor resection,ligamentous reconstruction, osteoplasty, biopsy and decompression,rodding of long bones such as the femur, tibia and humerus, placement ofscrews or other fixation devices in percutaneously or in an openprocedure, plate fixation and ring/external fixator placement or thelike.

At least some of these medical/surgical procedures typically requirepre- or intra-operative imaging guidance, such as fluoroscopic (x-ray)imaging, in order to achieve precise placement of the fixation device ina well-accepted location. This technology can simplify these proceduresby displaying the ideal entry point and placement of the fixationdevices as an AR or MR overlay over the anatomic/surgical site on thepatient.

In one non-limiting example, the system 100 may assist the surgeon 102in the fixation of an intertrochanteric hip fracture, which is commonlytreated with a hip cephallomedullary nail. Correct position of the nailin the femur (thigh bone) and the lag screw placed in the femoral headdirectly correlates with success of the implant and patient outcomes.The optimal implant position is attained by inserting the nail via thecorrect entry point in the femur, and the subsequent ideal placement ofthe screw in the femoral head. To obtain the correct entry point andorientation, a wire has to be passed through the correct location on thefemur. The system 100 may be utilized to predetermine the entry point ofthe guide wire and ensure accurate wire positioning and subsequent, nailand screw position.

It is to be noted that although the abovementioned medical proceduresare related to orthopaedic procedures, the system 100 may be utilized inother medical/surgical procedures in which a virtual display ofinformation relating to the procedure may be useful to the user of thesystem.

Developers have noted that certain MR techniques for displayinginformation to users during a medical procedure exist but rely on asingle communication route to gather the data and provide the data to acomputer system for further processing. In doing so, the conventionaltechniques may encounter various issues such as reduction or lack ofaccuracy, stability and reliability in case of interruption in thesingle communication route. This can have dangerous and potentiallyfatal outcomes. To this end, Developers have developed systems andmethods that can reduce or overcome the problems associated with theconventional techniques.

To improve the shortcomings of the conventional techniques, in variousnon-limiting embodiments of the present disclosure, the system 100 maycomprise a communication network with multiple communication routes.More specifically, the communication network may include at least tworeaders 110-1 and 110-2 that independently communicate with theplurality of elements 106-1, 106-2, 106-3, and 106-4 to gather the data.The readers 110-1 and 110-2 may be configured to directly communicateusing the communication link 120 with the computer system 114 to providethe gathered data. In addition, in certain non-limiting embodiments, thereaders 110-1 and 110-2 may be configured to wirelessly communicate withthe computer system 114 via the WSN router 112. Thus, the communicationnetwork comprising at least two readers 110-1 and 110-2 in place of oneand an additional communication layer of the WSN router 120 may providemultiple copies of the gathered data to the computer system 114 forprocessing and thereby ensuring a more accurate, stable and reliablesystem.

FIG. 2A depicts a high-level block diagram of components of the computersystem 114, in accordance with various embodiments of the presentdisclosure. It should be appreciated that FIG. 2A provides only anillustration of one implementation of the computer system 114 and doesnot imply any limitations with regard to the environments in whichdifferent embodiments may be implemented. Various modifications to thedepicted environment are possible to implement the computer system 114without departing from the principles presented herein. The computersystem 114 may be a server, a desktop computer, a laptop computer, orany device that may be configured to implement the present technology,as understood by persons skilled in the art.

As shown, the computer system 114 employs one or more different types ofprocessors 202, one or more computer-readable random access memories(RAMs) 204, one or more computer-readable read only memories (ROMs) 206,one or more computer-readable storage media 208, device drivers 214, aread/write (R/W) driver interface 216, a network interface 218, allinterconnected over a communication fabric 220. The communication fabric220 may be implemented by any architecture designed for communicatingdata and/or control information between processors (such asmicroprocessors, communications and network processors, etc.), systemmemory, peripheral devices, and any other hardware components within asystem.

The processor 202 of the computer system 114 may include one or more ofa CPU, an accelerator, a microprocessor, a GPU, an NPU, an ASIC, a FPGA,a dedicated logic circuitry, a dedicated artificial intelligenceprocessor unit, or combinations thereof.

One or more operating systems 210 and one or more application programs212 (examples of application programs may include programminginstructions) are stored on one or more of computer-readable storagemedia 208 for execution by one or more of the processors 202 via one ormore of respective RAMs 204 (which typically include a cache memory). Inthe illustrated embodiment, each of the computer-readable storage media208 may be embodied as a magnetic disc storage device of an internalhard drive, CD-ROM, DVD, memory stick, magnetic tape, magnetic disk,optical disk, a semiconductor storage device such as RAM, ROM, EPROM,flash memory or any other computer-readable tangible storage device thatcan store a computer program and digital information.

The R/W driver interface 216 reads from and writes to one or moreportable computer-readable storage media 226. The application programs212 may be related to the generating of the MR display of informationfor clinical use and stored on one or more of portable computer-readablestorage media 226, read via the respective R/W driver interface 216 andloaded into the respective computer-readable storage media 208.

Further, network interface 218 may be based on a TCP/IP adapter card orwireless communication adapter (such as a wireless communication adapterusing OFDMA technology). The application programs 212 on the computersystem 114 may be downloaded to the computer system 114 from an externalcomputer or an external storage device via a communication network 230(for example, the Internet, a local area network or other wide areanetwork or wireless network) and network interface 218. From the networkinterface 218, application programs 212 may be loaded onto thecomputer-readable storage media 208. The computer system 114 may connectto routers, firewalls, switches, gateway computers and/or edge serversof the communication network using copper wires, optical fibers,wireless transmission, and the like.

The computer system 114 may also include a display screen 222, akeyboard or keypad 224, and a computer mouse or touchpad 228. The devicedrivers 214 may interface with the display screen 222 for imaging, withthe keyboard or the keypad 224, with the computer mouse or the touchpad228, and/or with the display screen 222 (which may be a touch sensitivedisplay) for alphanumeric character entry and user selections. Thedevice drivers 214, R/W driver interface 216 and network interface 218may comprise hardware and software (stored on the computer-readablestorage media 208 and/or the ROM 206).

FIG. 2B depicts a high-level functional block diagram of the server 116of the system 100 of FIG. 1, in accordance with various embodiments ofthe present disclosure. In the depicted embodiment, the server 116 maybe embodied as a physical machine (e.g., a physical server) or a virtualmachine (e.g., a virtual server) that executes application programs toenable computer systems, such as the computer system 114 to communicatewith the server 116. The server 116 may include a processor 252, amemory 254, and a network interface 256. It is to be noted that theserver 116 may include other components which have not been illustratedfor the purpose of simplicity.

The processor 252 of the server 116 may include one or more of a CPU, anaccelerator, a microprocessor, a GPU, an ASIC, a FPGA, a dedicated logiccircuitry, a dedicated artificial intelligence processor unit, orcombinations thereof.

The memory 254 may include volatile memory (e.g., RAM) and non-volatileor non-transitory memory (e.g., a flash memory, magnetic storage, and/ora ROM). The non-transitory memory(ies) stores a platform that controlsthe overall operation of server 116. The platform, when executed byprocessor 252, implements application programs related to the generatingthe MR display of information for clinical use.

The network interface 256 may include one or more wireless transceiversconfigured for wireless communications with the communication network230, or one or more network adaptors configured for wired communicationswith the communication network 230. In general, the network interface256 may be configured to correspond with the network architecture ofthat is used to implement a link for communications between the server116 and the communication network 230. In certain embodiments, thenetwork interface 256 may be implemented in a similar manner as thenetwork interface 218.

It is to be noted that the server 116 is shown as a standalone computer.However, the implementation of various other embodiments of the presentdisclosure may include any client-server model where computer systemsmay run a client version of the application programs related to thegenerating the AR and/or MR display of information for clinical use.Other examples of the server 116 may include a distributed computingsystem that runs the server version of the application programs relatedto generating AR and/or MR display of information for clinical use, avirtual machine (or virtual machines) instantiated by the infrastructureof a public or private cloud, or a cloud service provider that providesthe application programs related to the generating the AR and/or MRdisplay of information for clinical use as a service (SaaS). Suchimplementations or any other similar implementation should not limit thescope of the present disclosure.

FIG. 3 illustrates a flowchart of a method 300 implemented on the system100 for generating an MR display of information for clinical use, inaccordance with various non-limiting embodiments of the presentdisclosure.

Step 302: Receiving Data, by the Processor, from a Communication NetworkComprising: A First Reader, a Second Reader and a Wireless SensorNetwork (WSN) Router

The method 300 commences at step 302 where the processor 202 of computersystem 114 receives the data from the communication network. Thecommunication network comprises: the readers 110-1 and 110-2 and the WSNrouter 112. Referring to FIG. 1, the readers 110-1 and 110-2 may beconfigured to communicate with the plurality of elements 106-1, 106-2,106-3, and 106-4 to read the received data associated with the pluralityof elements 106-1, 106-2, 106-3, and 106-4.

In certain non-limiting embodiments, the plurality of elements 106-1,106-2, 106-3, and 106-4 may be positioned at a given location relativeto the medical instruments 108 and/or the patient 104. For example, theelements 106-1 and 106-2 may be attached to an operating/proceduraltable 109 on which the patient 104 may be lying for the purposes of themedical procedure. In other embodiments, instead of anoperating/procedural table 109, the elements 106-1 and 106-2 may beattached or otherwise positioned on another type of patient support,such as a wheel chair or a bed. In some examples, the element 106-3 maybe disposed on the patient 104 and the element 106-4 may be attached tothe medical instrument 108. It is to be noted that the system 100 mayinclude additional elements, which are not illustrated for the purposeof simplicity.

In some of the examples, at least some of the plurality of elements106-1, 106-2, 106-3, and 106-4 may be RF markers (e.g., RFidentification (RFID) tags) while other elements may be optical markers.In some embodiments, all of the plurality of elements 106-1, 106-2,106-3, and 106-4 may be RF markers. In some embodiments, all of theplurality of elements 106-1, 106-2, 106-3, and 106-4 may be opticalmarkers.

In embodiments in which at least some of the plurality of elements106-1, 106-2, 106-3, and 106-4 are RF markers, the readers 110-1 and110-2 may be RF readers configured to independently communicate with theplurality of elements 106-1, 106-2, 106-3, and 106-4 to receive theassociated data. By way of example, the readers 110-1 and 110-2 maytransmit RF signals towards the plurality of elements 106-1, 106-2,106-3, and 106-4. In return, the plurality of elements 106-1, 106-2,106-3, and 106-4 may reflect the associated data (such as preassignedidentification).

In embodiments in which at least some of the plurality of elements106-1, 106-2, 106-3, and 106-4 are optical markers, the readers 110-1and 110-2 may be optical sensors configured to independently communicatewith the plurality of elements 106-1, 106-2, 106-3, and 106-4 to receivethe associated data. By way of example, the readers 110-1 and 110-2 maytransmit light signals towards the plurality of elements 106-1, 106-2,106-3, and 106-4 to read the associated data. In certain non-limitingembodiments, the associated data may be in the form of bar codes.

It is to be noted that a type of the elements 106-1, 106-2, 106-3, and106-4 may depend upon requirements of the system 100 and/or the clinicaluse. For example, if at least some of the plurality of elements 106-1,106-2, 106-3, and 106-4 may not remain in line of sight with the readers110-1 and 110-2 during the medical procedure, then such elements may beRF markers and may be attached to the medical instrument 108 and thepatient 104, for example. On the other hand, if some of the plurality ofelements 106-1, 106-2, 106-3, and 106-4 may remain in line of sightduring the medical procedure with the readers 110-1 and 110-2, then suchelements may be optical markers. Such elements may be attached to theoperating/procedural table 109. It is to be noted that above discussedarrangements of the plurality of elements 106-1, 106-2, 106-3, and 106-4are for the purposes of examples only and any other arrangement may beadapted without limiting the scope of present disclosure.

As previously discussed, different medical instruments 108 to be trackedand/or tracked targets such as the patient 104 may be provided with arespective set of the plurality of elements 106-1, 106-2, 106-3, and106-4 in different configurations. Differentiation of different medicalinstruments 108 and/or targets and their corresponding virtual volumesmay be possible based on the specification configuration and/ororientation of the different sets of plurality of elements 106-1, 106-2,106-3, and 106-4 relative to one another, enabling each such medicalinstrument 108 and/or other target to have a distinct individualidentity within the system 100. The individual elements may provide datato the computer system 114 via the readers 110-1 and 110-2 and the WSNrouter 112, such as data related to the size and/or shape of the trackedmedical instrument 108 within the system 100. The elements may alsoprovide additional information such as a specific position on thetarget, such as a central point or a tip of the medical instrument 108or a central axis of the medical instrument 108, among otherinformation. The plurality of elements 106-1, 106-2, 106-3, and 106-4may be tracked relative to a reference point or reference object in theoperating room, such as one or more reference points on the patient 104.

In certain non-limiting embodiments, the readers 110-1 and 110-2 mayinclude optical sensors (e.g., an infrared, LIDAR, depth and/or anyother optical sensor) arranged at various locations in the surgicalfield. For example, optical sensor locations may include one or more of:worn by the clinicians (such as surgeon, technician, nurse, resident, oranesthesiologist), or arranged at discrete static locations such as overthe surgical field, adjacent a display within the surgical field, etc.).In this embodiment, the readers 110-1 and 110-2 may be configured toscan a region of interest (ROI) based on communication with theplurality of elements 106-1, 106-2, 106-3, and 106-4. By way of example,after establishing a communication and determining a position and or anorientation of the plurality of elements 106-1, 106-2, 106-3, and 106-4,the readers 110-1 and 110-2 may scan the determined position and theorientation. In certain non-limiting embodiments, along with the datagathered from the plurality of elements 106-1, 106-2, 106-3, and 106-4,the readers 110-1 and 110-2 may provide the optical scan captured by theassociated optical sensors to the computer system 114.

In certain non-limiting embodiments, the readers 110-1 and 110-2 may beconfigured to independently and directly communicate with the computersystem 114 to forward the data gathered from the plurality of elements106-1, 106-2, 106-3, and 106-4. Also, in certain non-limitingembodiments, the readers 110-1 and 110-2 may be configured tocommunicate with the computer system 114 indirectly via the WSN router112 to forward the data gathered from the plurality of elements 106-1,106-2, 106-3, and 106-4. Thus, the computer system 114 may be configuredto receive the data from multiple sources and via multiple communicationroutes.

More specifically, the various communication routes can be described as:(i) the plurality of elements 106-1, 106-2, 106-3, and 106-4 to thereader 110-1 and from the reader 110-1 to the computer system 114, (ii)the plurality of elements 106-1, 106-2, 106-3, and 106-4 to the reader110-2 and from the reader 110-2 to the computer system 114, (iii) theplurality of elements 106-1, 106-2, 106-3, and 106-4 to the reader110-1, from the reader 110-1 to the WSN router 112, and from the WSNrouter 112 to the computer system 114, and (iv) the plurality ofelements 106-1, 106-2, 106-3, and 106-4 to the reader 110-2, from thereader 110-2 to the WSN router 112, and from the WSN router 112 to thecomputer system 114.

In so doing, in case any of the communication routes are broken due to acommunication link 120 break, the computer system 114 may still receivethe data from another communication route which is not broken. Forexample, if the communication link 120 is broken between one of thereaders 110-1 and 110-2 and either the WSN router 112 or the computersystem 114, the computer system 114 may still receive the data gatheredfrom the plurality of elements 106-1, 106-2, 106-3, and 106-4 via theother reader that is still communicating with the WSN router 112 and/orthe computer system 114. Thus, ensuring a more accurate, stable andreliable system 100 in certain embodiments.

In certain non-limiting embodiments, the computer system 114 may beconfigured to determine if the data has been received from one or moreof: the reader 110-1, the reader 110-2, and the WSN router 112. In otherwords, the computer system 114 is configured to determine whichcommunication route the data has been received from.

Further, in certain non-limiting embodiments, the computer system 114may be configured to interrogate the WSN router 112, the reader 110-1and the reader 110-2 to determine if any of the communication routes orcommunication links from the plurality of elements 106-1, 106-2, 106-3,and 106-4 was compromised or interrupted. To do so, in certainnon-limiting embodiments, the computer system 114 may be configured totransmit handshaking signals to the WSN router 112, the reader 110-1 andthe reader 110-2. In return, the WSN router 112, the reader 110-1 andthe reader 110-2 may transmit acknowledgment signals respectively toconfirm an establishment of the respective communication routes. Incase, any one of the WSN router 112, the reader 110-1 and the reader110-2 fails to provide the acknowledgment signal, the computer system114 may determine that the associated communication route may becompromised or interrupted. To this end, the computer system 114 maystill rely on other, non-compromised communication routes to receive thedata.

FIG. 4 illustrates at least some of the modules implemented on thecomputer system 114, in accordance with various non-limited embodimentsof the present disclosure. As shown, the computer system 114 may includea machine learning algorithm (MLA) module 402 and an MR module 404. TheMLA module 402 may include different trained MLA algorithms to performdifferent functionalities. The MR module may act as an interface for thedisplay 118 and may control the content to be displayed on the display118. It is to be noted that the computer system 114 may include othercomponents and modules, however, such components and modules have beenomitted from FIG. 4 for the purpose of simplicity.

In case the data has been received from more than one communicationroute (i.e., by more than one of the reader 110-1, the reader 110-2, andthe WSN router 112), the MLA module 402 may be configured to filter thedata to obtain filtered data. In other words, the MLA module 402 may beconfigured to filter the received data to remove the redundant data. Incertain non-limiting embodiments, the filtering of data may includedisregarding duplicate data based on a predetermined rule.

In certain non-limiting embodiments, the predetermined rule may includea predetermined hierarchy of communication routes. By way of example,the predetermined hierarchy of communication routes may be, indescending order: (i) the plurality of elements 106-1, 106-2, 106-3, and106-4 to the reader 110-1 and from the reader 110-1 to the computersystem 114, (ii) the plurality of elements 106-1, 106-2, 106-3, and106-4 to the reader 110-2 and from the reader 110-2 to the computersystem 114, (iii) the plurality 106-1, 106-2, 106-3, and 106-4 to thereader 110-1, from the reader 110-1 to the WSN router 112, and from theWSN router 112 to the computer system 114, and (iv) the plurality 106-1,106-2, 106-3, and 106-4 to the reader 110-2 from the reader 110-2 to theWSN router 112, and from the WSN router 112 to the computer system 114.In other embodiments, the predetermined hierarchy of communicationroutes may differ.

In certain non-limiting embodiments, the MLA module 402 may beconfigured to disregard the duplicate data associated with thecommunication route which is lower in the predetermined hierarchy. Inother words, the MLA module 402 may prioritize disregarding theduplicate data received from the communication route: the plurality ofelements 106-1, 106-2, 106-3, and 106-4 to the reader 110-2 from thereader 110-2 to the WSN router 112, and from the WSN router 112 to thecomputer system 114 over other communication routes.

In certain non-limiting embodiments, the predetermined rule may be basedon a predetermined hierarchy of time of transmission of the data fromthe plurality of elements 106-1, 106-2, 106-3, and 106-4. The mostrecently transmitted data may be prioritized for data processing whilethe data transmitted later in time may be removed by the MLA module 402through the filtering step.

In certain non-limiting embodiments, the predetermined rule may be basedon a predetermined hierarchy time of reception of the data by thecomputer system 114. The most recently received data may be prioritizedfor data processing while the data received later in time may befiltered by the MLA module 402.

The predetermined rule may be based on any other hierarchy relating tothe data transmission. Any of the predetermined rules may be stored in amemory, such as the memory 254.

Step 304: Based on the Received Data, Determining at Least One of aPosition and an Orientation of a Given Element of the Plurality ofElements

The method 300 proceeds to step 304 where, based on the received data,the processor 202 of computer system 114 determines at least one of aposition and an orientation of a given element of the plurality ofelements 106-1, 106-2, 106-3, and 106-4.

In certain non-limiting embodiments, the plurality of elements 106-1,106-2, 106-3, and 106-4 may communicate with the readers 110-1 and 110-2to give identifiable points for tracking the patient 104, and themedical instrument 108. Based on the identifiable points, the computersystem 114 may determine at least one of a position and an orientationof a given element of the plurality of elements 106-1, 106-2, 106-3, and106-4.

In certain non-limiting embodiments, the medical/surgical instrument 108may be defined by a grouping of at least some of the plurality ofelements 106-1, 106-2, 106-3, and 106-4, which may define at least apart of a rigid body of the medical/surgical instrument 108. Thus, basedon the determined position and/or the orientation of the given elementof the plurality of elements 106-1, 106-2, 106-3, and 106-4, thecomputer system 214 may determine the position and/or orientation in ofthe medical instrument 108 in a 3D virtual space. The position and/ororientation of the medical instrument 108 in 3D may be tracked in sixdegrees of freedom (e.g., x, y, z coordinates and pitch, yaw, rollrotations), in five degrees of freedom (e.g., x, y, z, coordinate andtwo degrees of free rotation), but typically tracked in at least threedegrees of freedom (e.g., tracking the position of the tip of themedical instrument 108 in at least x, y, z coordinates).

In a similar manner, the computer system 114 may track a position and/ororientation of the patient 104 based on the determined position and/orthe orientation of the given element of the plurality of elements 106-1,106-2, 106-3, and 106-4.

Referring to FIG. 4, in certain non-limiting embodiments, the MLA module402 may be configured to perform a comparative analysis of the receiveddata to ensure that the location and/or orientation of the plurality ofelements 106-1, 106-2, 106-3, and 106-4 are determined in an optimizedand accurate manner. The precise location may be determined by receivingthe data from several communication routes and calculating the variousmeasurements to determine position and/or orientation. In onenon-limiting embodiment, the position and/or orientation of theplurality of elements 106-1, 106-2, 106-3, and 106-4 may be determinedby a process referred to as triangulation. The MLA module 402 maytriangulate the data and compare the readings, then use the resultingdata to communicate the location to the MR module 404.

As previously noted, the plurality of elements 106-1, 106-2, 106-3, and106-4 may be attached to the different objects (e.g., the patient 104,the medical/surgical instrument 108, and the operating/procedural table109). In certain non-limiting embodiments, at least some of theplurality of elements 106-1, 106-2, 106-3, and 106-4, for example theelements 106-1 and 106-2, may be positioned in a fixed manner withrespect to the readers 110-1 and 110-2. Such fixed elements may be usedas reference points to determine a position of the other elements, suchas the elements 106-3 and 106-4, that may be mobile with respect to theother elements 106-1 and 106-2 and the readers 110-1 and 110-2.

FIG. 5 illustrates a representative technique of triangulation performedby the computer system 114 in accordance with various non-limitingembodiments of the present disclosure. The triangulation approach,illustrated in FIG. 5, comprises measuring the angle of incidence of atleast two elements with respect to the reader 110-1. One out of the twoelements 106-2 and 106-3 may be fixed with respect to the reader 110-1and the distance of the fixed element, such as element 106-2 may beknown prior to performing the triangulation. The estimated position ofthe element 106-3 may correspond to the intersection of the linesdefined by angle 1 and angle 2.

It is to be noted that in FIG. 5 triangulation is illustrated using twoelements 106-2 and 106-3. However, in various non-limiting embodiments,more than two elements may be involved in triangulation without limitingthe scope of present disclosure. Further, a similar triangulationbetween the plurality of elements 106-1, 106-2, 106-3, and 106-4 and thereader 110-2 may be performed by the computer system 114.

It is to be noted that in the above discussion, the MLA module 402determines the position and/or orientation of the plurality of elements106-1, 106-2, 106-3, and 106-4. However, in other non-limitingembodiments, the computer system 114 may rely on any suitable techniqueor other module to determine the position and/or orientation of theplurality of elements 106-1, 106-2, 106-3, and 106-4.

As previously noted, in addition to the data gathered from the pluralityof elements 106-1, 106-2, 106-3, and 106-4, in certain non-limitingembodiments, the readers 110-1 and 110-2 may capture an optical scan ofthe ROI as well. The readers 110-1 and 110-2 may provide the opticalscans directly to the computer system 114 or via the WSN router 112 in asimilar manner as discussed above.

The computer system 114 may be configured to stitch together the opticalscans to generate a three-dimensional (3D) panoramic image of the ROI.In certain non-limiting embodiments, the computer system 114 may beconfigured to render the 3D image onto the display 118.

In certain non-limiting embodiments, the computer system 114 maytransform the optical scans into a first set of 3D color point clouds;and combine the first set of 3D color point clouds into a composite 3Dcolor point cloud depicting the ROI. Based on the composite 3D colorpoint cloud, the computer system 114 may then determine a positionand/or orientation of the medical instrument 108 with respect to thepatient 104.

The computer system 114 may additionally or alternatively computedistance data, such as in the form of a 3D point cloud output by a LIDARsensor arranged over the operating/procedural table 109. The computersystem 114 may further merge digital photographic color images withdistance data to generate a substantially dimensionally accurate colormap of the ROI within the operating room.

Step 306: In Response to the Determined at Least One of a Position andan Orientation, Causing to be Displayed, on a Display CommunicativelyCoupled to the Computer System, Predetermined Information Associatedwith the Determined at Least One of a Position and an Orientation

The method 300 proceeds to step 306 where, in response to the determinedat least one of a position and an orientation of a given element of theplurality of elements 106-1, 106-2, 106-3, and 106-4, the processor 202of computer system 114 causes to display, on the display 118communicatively coupled to the computer system 114, predeterminedinformation associated with the determined at least one of a positionand an orientation.

The predetermined information may include but not limited to augmentedstill/video images, medical procedure guidance, navigation menu, eyefatigue indicator, surgeon's profile, annotations or the like, detailsof which are discussed below.

Referring to FIG. 4, in certain non-limiting embodiments, afterdetermining the position and/or orientation of the given element of theplurality of elements 106-1, 106-2, 106-3, and 106-4, the MLA module 402may be configured to provide the position and/or orientation to the MRmodule 404. The position and the orientation may act as a trigger forthe MR module 404 to initiate and control the predetermined informationto be displayed on the display 118.

As previously discussed, the system 100 may include the display 118. Thedisplay 118 may be used during a medical procedure to project or displaythe predetermined information during the medical procedure to augmentthe surgeon's vision. The display of predetermined information may bebased on the determined position and the orientation of a given elementof the plurality of elements 106-1, 106-2, 106-3, and 106-4. The MRmodule 404 may be based on MARGE™ platform, for example. MARGE™ platformis an MR content authoring and delivery platform that may beincorporated in the MR module 404. The MARGE™ platform may assist the MRmodule 404 to act as a platform between the MLA module 402 and thedisplay 118.

In some non-limiting embodiments, the computer readable storage media208 may be configured to store various still/video images to bedelivered to the display 118. For example, the images may includeinformation useful to the clinician for the medical procedure and setup.In other non-limiting embodiments, the memory 254 associated with theserver 116 may be configured to store the various still/video images tobe delivered to the display 118, such as for the pre-operation medicalprocedure and setup. In any case, the MLA module 402 may be configuredto fetch the still/video images either from the computer readablestorage media 208 or from the memory 254.

In certain non-limiting embodiments, the system 100 may includeadditional cameras (not illustrated) to capture still/video images ofthe at least a portion of the patient 104. In embodiments in which thedisplay 118 is a wearable device, such cameras may be integrated withthe wearable device. In other embodiments, such cameras may be locatednear to the patient 104, or be positioned relative to the patient tocapture images of the patient. The computer system 114 may be configuredto receive the still/video images captured by the additional cameras,such as, in the form of optical scans as previously discussed. Thecomputer system 114 may be configured to stitch together the opticalscans to generate a three-dimensional (3D) panoramic image.

The MLA module 402 may be configured to provide all the still/videoimages fetched from the computer readable storage media 208 or from thememory 254 or captured by the additional cameras to the MR module 404.In response to the determined position and/or the orientation of a givenelement of the plurality of elements 106-1, 106-2, 106-3, and 106-4, theMR module 402 may be configured to augment virtual objects over thestill/video images.

The MR module 404 may be configured to transmit the augmentedstill/video images to the display 118 at the determined position and/orthe orientation of a given element of the plurality of elements 106-1,106-2, 106-3, and 106-4 and may integrate it using multi-sensor datafusion relative to the surrounding data collected from the sensors, suchas sensors on the wearable device.

In certain non-limiting embodiments, the MR module 404 may be configuredto determine a navigational guide, along with actionable digitalelements such as a digital button to be displayed on the display 118 toassist the surgeon 102 to perform certain medical procedures such asinserting protheses, implants, guides, anchorage devices and othersurgical devices which would be part of the medical procedure. Thenavigational guide along with the actionable digital elements mayimprove the interaction control of the surgeon 102 over the display 118.It is to be noted that the MR virtualizations may be streamed at thedesired position and orientation via server 116, cloud services, or thelike without limiting the scope of present disclosure.

In certain non-limiting embodiments, the MR module 404 may provide anability to incorporate spatial gesture recognition. The spatial gesturesmay include but are not limited to movement of eyes, head, arms, handsetc. It is to be noted that the tracking of the spatial gesture may beperformed by any suitable technique, for example, computer vision. Priorto the transmitting the augmented still/video images, the MR module 404may provide a visual pointer displayed on the display 118. The visualpointer may be moved (through spatial gesture recognition) to thedesirable spatial viewpoint at which the augmented still/video imagesmay be streamed.

In certain non-limiting embodiments, the navigational guide may have anavigation menu that may allow the surgeon 102 to properly select theimage related to the intended medical/surgical procedure for example,via spatial gesture movement. The navigation menu may reflectgamification functionality which may engage the surgeon 102 during themedical procedure. Such functionality may further include aspectsrelating to surgical ergonomics that may reduce discomfort and mitigatenegative downstream consequences.

In certain non-limiting embodiments, the MR module 404 may provide anindicator on the display 118. The indicator may provide an indication ofeye fatigue warning. The indicator may comprise a power bar-type meterrepresentation which may be located on the display 118 in the field ofview of the surgeon 102. The power in the bar may be determined byeye-level postural deviations from normal values associated with thegiven surgeon 102 or typical base line values.

In certain non-limiting embodiments, the computer system 114 may allow aregistration of the physician/surgeon's profile on a network such as theserver 116. The registration may include registering physicians/surgeonsin an online application by saving their credentials and preferences ina database, such as the memory 254. The registration process may assistthe MR module 404 to display a physician's/surgeon's preferencesrelating to the predetermined information such as sizes, angles andviews of images and text, medical/surgical preferences and preferredmedical/surgical instruments 108 (e.g. tools that contain sensors). Thephysician/surgeon preferences may be manually input or be determined bythe AI module. In another example, the surgeon's profile would allow forproviding rights and privileges, authorization, and capturing timestamps.

In certain non-limiting embodiments, the MR module 404 may provideannotating the displayed image. The physician/surgeon 102 may annotatevirtual or real objects on the display 118 for example, thephysician/surgeon may change the angle of a rod insertion relative to areference angle as provided by the MLA module 402. By fixing annotationsto a spatial location, the MR module 404 may enable the surgeon toannotate their MR experience in the virtual space. An annotationdisplayed in the virtual space remains readable at all viewing anglesand distances.

The process of annotating may allow the physician/surgeon 102 to quicklydefine a placement using a spatial gesture input, providing a benefit oflocalization and data accessibility. In certain non-limitingembodiments, the MR module 404 may provide annotating a procedure, or anobject by determining where the annotation is located in the physicalenvironment and by adding a visual gesture recognizer to the view.

Annotation may also provide an overlay of a surgical plan allowing formark-up to represent any deviation that the surgeon 102 wishes tointroduce relative to the best practice. In certain non-limitingembodiments, the MR module 404 may provide a remote assistance feature,though which the physician/surgeon 102 may consult with aknowledge-based individual for further assistance in the medicalprocedure.

FIG. 6 illustrates a representative display depicting predeterminedinformation overlaid over a patient's body part in accordance withvarious non-limiting embodiments of the present disclosure. FIG. 6illustrates an insertion point and an orientation of a retrograde nailto be inserted in the patient 104. A virtual surgical plan is overlaidover the corresponding anatomy of the patient 104 in real-time. Anoptimal location and orientation of the retrograde nail is indicated asthe predetermined information and/or the surgical plan.

FIG. 7 illustrates a flowchart of another method 500, according toaspects and embodiments of the present technology. The method 500 may beimplemented on the system 100 for generating a MR display of informationfor clinical use, in accordance with various non-limiting embodiments ofthe present disclosure.

Step 502: Receiving Data, by the Processor, from a Communication NetworkComprising: A First Reader and a Second Reader

The method 500 commences at step 502 where the processor 202 of thecomputer system 114 receives the data from the communication network.The communication network comprises readers 110-1 and 110-2. Aspreviously noted, the reader 110-1 is communicatively coupled to, andconfigured to receive data from, each element of a plurality of elements106-1, 106-2, 106-3, and 106-4, each element positioned at a givenlocation relative to an object at a clinical site. Similarly, the reader110-2 is communicatively coupled to, and configured to receive datafrom, each element of the plurality of elements 106-1, 106-2, 106-3, and106-4, each element positioned at a given location relative to an objectat a clinical site. Optionally, the communication network may alsoinclude the WSN router 112. The processor 202 of computer system 114 mayreceive the data from a WSN router 112 in addition to the readers 110-1and 110-2. The WSN router 112 may be communicatively coupled to, andconfigured to receive data from, each of the readers 110-1 and 110-2.

Step 504: Determining, by the Processor, if Data has been Received fromOne or More of the First Reader, and the Second Reader

The method 500 proceeds to step 504 where the processor 202 of computersystem 114 determines if the data has been received from one or more ofthe readers 110-1 and 110-2. Optionally, the processor 202 of thecomputer system 114 may determine if the data has been received from theWSN router 112 as well.

Step 506: In Response to the Data Having been Received from More thanOne of the First Reader and the Second Reader, Filtering the Data toObtain Filtered Data, the Filtering Comprising Disregarding DuplicateData

The method 500 proceeds to step 506 where in response to the data havingbeen received from more than one of the readers 110-1 and 110-2, theprocessor 202 of the computer system 114 may be configured to filter thedata to obtain filtered data. As previously noted, the step of filteringcomprises disregarding duplicate data. As previously noted, inembodiments in which the data has been received from more than one ofthe reader 110-1, the reader 110-2, the MLA module 402 may be configuredto perform filtering the data to obtain filtered data. In other words,the MLA module 402 may be configured to perform filtering of thereceived data to remove the redundant data. In certain non-limitingembodiments, the filtering of data may include disregarding duplicatedata based on a predetermined rule (discussed previously).

Step 508: Determining, Based on the Filtered Data, at Least One of aPosition and an Orientation of a Given Element of the Plurality ofElements

The method 500 proceeds to step 508 where based on the filtered data,the processor 202 of computer system 114 determines at least one of aposition and an orientation of a given element of the plurality ofelements 106-1, 106-2, 106-3, and 106-4. It is to be noted that theprocess of determining at least one of the position and the orientationof the given element of the plurality of elements 106-1, 106-2, 106-3,and 106-4 may performed in a similar manner as previously discussed withrespect to step 304 of the method 300.

Step 510: In Response to the Determined at Least One of a Position andan Orientation, Cause to be Displayed, on a Display CommunicativelyCoupled to the Computer System, Predetermined Information Associatedwith the Determined at Least One of a Position and an Orientation

The method 500 proceeds to step 510 where in response to the determinedat least one of the position and the orientation of a given element ofthe plurality of elements 106-1, 106-2, 106-3, and 106-4, the processor202 of computer system 114 causes to display, on the display 118communicatively coupled to the computer system 114, predeterminedinformation associated with the determined at least one of a positionand an orientation. It is to be noted that the process of displaying onthe display 118 may performed in a similar manner as previouslydiscussed with respect to step 306 of the method 300. By virtue ofmultiple routes to transmit the data from the plurality of elements106-1, 106-2, 106-3, and 106-4 to the computer system 114, the system100 provides stable, accurate and reliable localization of the MRcontent, in certain embodiments.

Network redundancy (i.e., multiple routes) adds additional instances ofnetwork components and lines of communication to ensure dataavailability and decrease a risk of failure along the critical dataroutes. When one communication route is unavailable, an alternatecommunication route may be instantly deployed to ensure minimal downtimeand continuity of network services thus improving the reliability. Byadding targeted complexity, the system 100 reduces the probability thata failure will take the system down. In addition to network redundancy,the system 100 also provide data redundancy, thereby increasing theaccuracy of the data.

It is to be understood that the operations and functionality of system100, constituent components, and associated processes may be achieved byany one or more of hardware-based, software-based, and firmware-basedelements. Such operational alternatives do not, in any way, limit thescope of the present disclosure.

It will also be understood that, although the embodiments presentedherein have been described with reference to specific features andstructures, it is clear that various modifications and combinations maybe made without departing from such disclosures. The specification anddrawings are, accordingly, to be regarded simply as an illustration ofthe discussed implementations or embodiments and their principles asdefined by the appended claims, and are contemplated to cover any andall modifications, variations, combinations or equivalents that fallwithin the scope of the present disclosure.

What is claimed is:
 1. A method for displaying predetermined informationfor clinical use, the method executable by a processor of a computersystem, the method comprising: receiving data, by the processor, from acommunication network comprising: a first reader, a second reader and awireless sensor network (WSN) router: the first reader beingcommunicatively coupled to, and configured to receive data from, eachelement of a plurality of elements, each element positioned at a givenlocation relative to an object at a clinical site; the second readerbeing communicatively coupled to, and configured to receive data from,each element of the plurality of elements; the WSN routercommunicatively coupled to, and configured to receive data from, each ofthe first reader and the second reader; based on the received data,determining at least one of a position and an orientation of a givenelement of the plurality of elements; in response to the determined atleast one of the position and the orientation, causing to be displayed,on a display communicatively coupled to the computer system,predetermined information associated with the determined at least one ofthe position and the orientation.
 2. The method of claim 1 furthercomprising: determining if the data has been received from more than oneof: the first reader, the second reader and the WSN router; in responseto the data having been received from more than one of the first reader,the second reader and the WSN router, filtering the data to obtainfiltered data, the filtering comprising disregarding duplicate databased on a predetermined rule, the determining the at least one of theposition and the orientation being based on the filtered data.
 3. Themethod of claim 2, wherein determining if the data was received frommore than one of the first reader, the second reader and the WSN routercomprises the processor interrogating the first reader, the secondreader and the wireless sensor network (WSN) router to determine if anyone of communication routes between the plurality of elements and thecomputer system was compromised.
 4. The method of claim 2, wherein thepredetermined rule comprises one of: a predetermined hierarchy ofcommunication routes between the plurality of elements and the computersystem, a predetermined hierarchy of time of transmission of the data;and a predetermined hierarchy of time of reception of the data by thecomputer system.
 5. The method of claim 1, wherein the object is one ormore of: a patient, a medical/surgical instrument, andoperating/procedural table.
 6. The method of claim 1, whereindetermining at least one of the position and the orientation of thegiven element of the plurality of elements comprises triangulation. 7.The method of claim 6, wherein the triangulation is performed bycomputing angles of incidences between the plurality of elements and thefirst reader and the second reader.
 8. The method of claim 1, whereinthe display is a fixed display or is part of a wearable device.
 9. Themethod of claim 1, wherein the predetermined information is one or moreof: an augmented image, a medical procedure guidance, a navigation menu,an eye fatigue indicator, a user's profile, and an annotation.
 10. Themethod of claim 1, wherein displayed information comprises an actionabledigital element, based on an input received from a user, the actionabledigital element is configured to display of further informationassociated with the predetermined information.
 11. The method of claim1, wherein the processor is configured to retrieve the predeterminedinformation from a memory of the computer system.
 12. The method ofclaim 1, wherein the predetermined information which is displayed isbased on a user's profile.
 13. The method of claim 1, further comprisingtracking user's eye, head or arm movement for interaction with thedisplay.
 14. The method of claim 1, wherein the plurality of elementsare radio frequency based elements.
 15. The method of claim 1, whereinthe plurality of elements are optical based elements.
 16. A system fordisplaying predetermined information for clinical use, the systemcomprising a communication network including: a plurality of elementshaving data stored therein and positioned at a given location relativeto an object at a clinical site; a first reader and a second reader,each of the first reader and the second reader communicatively coupledto each element of the plurality of elements and configured to receivethe data from each element of the plurality of elements and transmit thedata to a computer system; a wireless sensor network (WSN) routerconfigured to receive the data from the first reader and the secondreader and transmit the data to the computer system; the computer systembeing independently communicatively coupleable with each of the firstreader, the second reader and the WSN router for receiving thetransmitted data independently from each of the first reader, the secondreader and the WSN router; wherein the computer system is configured toexecute a method comprising: receiving data from the communicationnetwork; based on the received data, determining at least one of aposition and an orientation of a given element of the plurality ofelements; in response to the at least one of the determined position andthe orientation, causing display, on a display communicatively coupledto the computer system, predetermined information associated with thedetermined the at least one of the determined position and theorientation.
 17. The system of claim 16, wherein the plurality ofelements are radio frequency based elements.
 18. The system of claim 16,wherein the plurality of elements are optical based elements.
 19. Amethod for displaying predetermined information for clinical use, themethod executable by a processor of a computer system, the methodcomprising: receiving data, by the processor, from a communicationnetwork comprising: a first reader and a second reader: the first readerbeing communicatively coupled to, and configured to receive data from,each element of a plurality of elements, each element positioned at agiven location relative to an object at a clinical site; the secondreader being communicatively coupled to, and configured to receive datafrom, each element of the plurality of elements, each element positionedat a given location relative to the object at the clinical site;determining, by the processor, if the data has been received from one ormore of the first reader and the second reader; in response to the datahaving been received from more than one of the first reader and thesecond reader, filtering the data to obtain filtered data, the filteringcomprising disregarding duplicate data; determining, based on thefiltered data, at least one of a position and an orientation of a givenelement of the plurality of elements; in response to the determined atleast one of the position and the orientation, cause to be displayed, ona display communicatively coupled to the computer system, predeterminedinformation associated with the determined at least one of the positionand the orientation.
 20. The method of claim 19, wherein thecommunication network further comprises a wireless sensor network (WSN)router, the WSN router communicatively coupled to, and configured toreceive data from, each of the first reader and the second reader, themethod further comprising receiving data from the WSN router.